Papers with MLE loss
Generating Temporally-ordered Event Sequences via Event Optimal Transport (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for temporal event ordering and event infilling ignore the global semantics of events, and the model adopts a word-level objective to model events in texts. |
| Approach: | They propose a temporal event ordering and event infilling task using a model that uses maximum likelihood estimation to model events in texts. |
| Outcome: | The proposed model outperforms existing models on all evaluation datasets. |
Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing language models generate repetitive texts with greedy decoding or beam search. |
| Approach: | They propose a self-contrastive training technique to penalize the output of a premature checkpoint of the same model when it incorrectly predicts repetition. |
| Outcome: | The proposed training mitigates repetition while maintaining fluency while minimizing the overestimation of token-level repetition probabilities. |